Data-driven models for timing feedback responses in a Map Task dialogue system

Traditional dialogue systems use a fixed silence threshold to detect the end of users’ turns. Such a simplistic model can result in system behaviour that is both interruptive and unresponsive, which in turn affects user experience. Various studies have observed that human interlocutors take cues from speaker behaviour, such as intonation, paralanguage, content, and gestures among others, to coordinate smooth exchange of speaking turns. However, hardly any effort has been made towards implementing these models in dialogue systems and verifying how well they model the turn-taking behaviour in human–computer interactions. In this paper, we present a data-driven approach to building models for online detection of suitable feedback response locations in the user’s speech. We first collected human–computer interaction data using a spoken dialogue system that can perform the Map Task with users (albeit using a trick). Using this data we trained various models that use automatically extractable prosodic, contextual and lexico-syntactic features for online detection of feedback response locations. Next, we implemented a trained model in the same dialogue system and evaluated it in interactions with users. The results of a perception test of the user interactions support our hypothesis that the trained model provides for smoother dialogue in contrast to a baseline model. We also found that the trained model enhances the system’s responsiveness in contrast to the baseline model. To our knowledge, this is the first work on actual verification of the proposals on using speaker behavioural cues such as prosody, syntax and context, in modelling human-like turn-taking behaviour in spoken dialogue systems. Our results confirm that a model trained on these speaker modalities offers both smooth turn-transitions and responsive system behaviour.